The present invention relates to computer-implemented methods for manipulating shape and color of images.
In many graphics applications, a special effect operation known as xe2x80x9cwarpingxe2x80x9d or xe2x80x9cmorphingxe2x80x9d is used to gradually transform one image into another image. This is accomplished by creating a smooth transitional link between the two images. Some computer programs, for example, use warping to generate an animation sequence using the image transformations. Such an animation might, for example, show a first person""s face transforming into a second person""s face.
The warping process preserves features associated with each image by mapping the features from a source image to corresponding features in a destination image. In particular, mesh warping warps a first image into a second image using a point-to-point mapping from the first image to the second image. A first lattice (mesh) is superimposed on the first image and second lattice is superimposed on the second image. For each point in the first lattice, a one-to-one correspondence with a corresponding point in the second lattice is defined. Mesh warping is generally described in George Wolberg, Digital Image Warping, IEEE Computer Society Press (1990). Variations on mesh warping include a version in which the user specifies lines on the first image corresponding to lines on the second image.
These user-specified lines are used to construct corresponding lattices, which are used to morph the first image into the second image, as described above.
It is also possible to perform warping by specifying a first closed contour (such as a circle, a square or other) in the first image and a corresponding second closed contour in the second image. Points inside the first closed contour are warped to points inside the second closed contour. For instance, a software package called Elastic Reality, available from Avid Technology, Inc., provides a shape-based interface in place of mesh-based controls. This software allows users to use drawing tools and Bezier curves to define and edit multiple areas of an image to be manipulated over time.
Different transformations exist for warping from a first shape to a second shape with varying displacements of points inside each shape. One type of transformation for the inside points is a transformation that preserves the angles between any two intersecting curves and thus allow small shapes inside warping area to remain unchanged. Although conventional software transforms one shape into another, they do not preserve the angles between intersecting curves.
On a parallel note, color in computer graphics is defined in terms of xe2x80x9ccolor spacesxe2x80x9d, which are related to real or imaginary display devices such as monitors and color printers. Various color spaces are used to represent color on computers. Each image is associated with a color space which defines colors according to a combination of properties. For example, in an RGB (Red Green Blue) color space, each color is represented by a combination of red, green, and blue components. In a CMYK (Cyan Magenta Yellow Black) color space, each color is represented as a combination of cyan, magenta, yellow, and black. The color space is represented as a multi-dimensional vector space such as a three-dimensional space. Each point in such a three-dimensional color space corresponds to a color having amounts of red, green, and blue corresponding to the point""s coordinates.
An output device such as a computer monitor or printer is capable of reproducing a limited range of colors. An output device""s xe2x80x9ccolor gamutxe2x80x9d is the set of colors that the output device is capable of reproducing. Similarly, the xe2x80x9cvisible color gamutxe2x80x9d is the set of colors that the human eye is capable of perceiving. Color gamuts can be represented as a two-dimensional projection of their three-dimensional representations onto the plane of constant luminance.
A color gamut can be represented graphically in a color space""s coordinate system by a chromaticity diagram which represents the color gamut as a shape enclosing the coordinates of the colors contained in the color gamut. The visible color gamut, for example, appears as a three-dimensional oval shape when displayed in most coordinate systems. The color gamut of a computer monitor is typically smaller than the visible color gamut. Because it is generated using three types of phosphor, the color gamut of a monitor typically appears as a triangular shape falling within the visible color space""s oval when displayed graphically.
Each pixel in a graphical image typically has a color associated with it. Some colors in an image may not fall within a particular output device""s color gamut, and therefore cannot be reproduced exactly on the output device. One approach to reproducing such colors on the output device is referred to as xe2x80x9cgamut compressionxe2x80x9d or xe2x80x9cgamut mapping.xe2x80x9d There is a point within the output device""s color gamut that represents the color produced by maximum intensity of the three primary colors (R,G,B) of the device. This color represents the balance between the three primaries of the real or imaginary device. It is usually close to white and is referred to as the xe2x80x9cwhite point.xe2x80x9d A straight line is constructed from the white point to the point representing the color to be reproduced. The color may then be mapped to the point at which the line intersects the boundary of the output device""s color gamut. This process is also called clipping. Alternatively, the line may be extended until it intersects with the visible color gamut. The line is then compressed by appropriate scaling so that it fits entirely within the output device""s color gamut. In this way, each point on the original (non-compressed) line is mapped to a corresponding point on the compressed line. This process is called desaturation and represents xe2x80x9cmoving towardxe2x80x9d the white point. For example, xe2x80x9csaturationxe2x80x9d is always xe2x80x9cmoving awayxe2x80x9d from the white point.
In this way, conventional color corrections are related to color spaces and explicitly use the white point of the corresponding color space. The reference to the white point over-emphasizes certain colors, namely those at or near the white point, and introduces unnatural, device-dependent color transformations in the color correction process.
In one aspect, a computer-implemented method for mapping a first area to a second area is disclosed. The first area is defined by a first enclosing contour and the second area is defined by a second enclosing contour. The method includes defining the first enclosing contour; modifying the first enclosing contour into the second enclosing contour; generating an analytic function to conformally warp the first area into the second area; and performing the conformal warp using the analytic function.
In a second aspect, a computer-implemented method maps a first color gamut to a second color gamut. In this aspect, the first color gamut is enclosed by a first enclosing contour and the second color gamut is enclosed by a second enclosing contour. The method includes defining the first enclosing contour; modifying the first enclosing contour into the second enclosing contour; generating an analytic function to conformally warp the first color gamut into the second color gamut; and performing the gamut mapping using the analytic function.
Advantages of the invention include one or more of the following. One advantage of the invention is that it provides a method for warping from a first image to a second image based on a first contour and a second contour, without requiring the user to define mappings from individual points within the fist contour to individual points within the second contour. Rather, the user needs to only specify the first and second contours and correspondences between them. This increases the ease of use with which the user can define a mapping between the first and second images and also allows for a more uniform warping which preserves angles.
Further, the ability to warp a first color gamut into a second color gamut is particularly advantageous. Such warping can be used to produce visually interesting effects such as producing an animation sequence in which a daytime scene is transformed into a sunset scene, for example.
Moreover, the use of contours rather than meshes to define mappings decreases the sophistication required of the user in using the invention. It is much easier and more intuitive for the user to draw shapes around corresponding areas in the first and second images or first and second color gamuts, rather than for the user to define point-to-point or color-to-color mappings.
In the case of color gamut warping, the graphical display of multiple color gamuts provides the user with the ability to quickly identify colors within the first color gamut that do not fall within the second color gamut. Specification of the mapping between the two color gamuts by drawing contours around graphical representations of the color gamuts provides the user with an intuitive interface to the warping process, mapping from one device gamut to a second device gamut.
Because using analytic functions to map a first color gamut to a second color gamut preserves the angles, it preserves the relationships among colors and the general appearance of the image after the transformation. In particular, the lines of constant hue and constant saturation are mapped into lines that can be interpreted as lines of constant hue and saturation because they remain orthogonal at the points of intersection after the transformation.